Conventional video interfaces are limited either by requiring physical user input, such as a person's hand, which may be used to interact with an input device such as a mouse, keyboard, trackpad, or touchscreen, or speech recognition, which may either understand the words the person is saying or which may determine a person's emotional affect or mental state in order to determine approval or disapproval for the displayed content. In the case of the former, a user must have a hand free to interact with the input method, or, in the latter, must be able to speak clearly in order to interact with a media presentation. With emotional affect or mental state voice recognition, there may not be any background noises that may register a false positive reading. This is often impossible in public settings such as on public transportation, at public events, at fitness centers, or even while a third-party is talking to the user. Also, for speech recognition, an artificial intelligence system must be able to understand the speaker, which may be difficult to record if a media presentation is playing simultaneously on the same device.
It is known to have a system for reading facial expressions that may categorize such expressions into seven different primary categories, such categories including happiness, sadness, contempt, disgust, surprise, fear, and anger. Such expressions may be identified by measuring changes in movement of certain facial features, namely the corners of the mouth, the corners of the eyes, the vertical position of the eyebrows, the prominence of the cheeks, and the furrows of the forehead.
It is known to have a facial recognition software which may be used on a smartphone, tablet, or computer, and which may read a plurality of facial landmarks on a user's face. Such a software may use such facial landmarks to read the position of and changes in position of various facial muscles associated with facial expressions. Such a software may read facial expressions in still photos or may read facial expressions in real time. Such readings may be displayed on a chart of measured landmarks over time to determine a correlation with the movements of particular facial muscles and their association with specific facial expressions.
It is known to have a facial recognition software which may be used to determine a progression of video sequences based on a user's facial expressions. Such a software may include information on one or more of a group consisting of facial expressions, action units, head gestures, smiles, brow furrows, squints, lowered eyebrows, raised eyebrows, smirks, and attention. The physiological data may include one or more of electrodermal activity, heart rate, heart rate variability, skin temperature, and respiration. The method may further comprise inferring of mental states based on the mental state data which was collected. The mental states may include one of a group consisting of frustration, confusion, disappointment, hesitation, cognitive overload, focusing, being engaged, attending, boredom, exploration, confidence, trust, delight, valence, skepticism, and satisfaction. The playing of the first media presentation may be done on a mobile device and further comprising recording of facial images with the mobile device as part of the capturing of the mental state data. Such a system may be known as “Video recommendation based on affect” and is disclosed in U.S. patent application Ser. No. 13/406,068.
There is a need in the art for a computer-implemented method and system for analyzing a user's facial responses to determine an appropriate progression of outputs. Such a system may provide an alternate input means allowing users to view and control media presentations when the user's hands may be full or when the user's voice may be unrecognizable to the presenting device. Such a system may be implemented for advancing media presentations based on user facial expressions, which may be prompted or impromptu, and may further comprise an eye-tracking feature to ensure that a user is viewing the media presentations.
It is to these ends that the present invention has been developed.